Search for: All records

Marine scientists have been leveraging supervised machine learning algorithms to analyze image and video data for nearly two decades. There have been many advances, but the cost of generating expert human annotations to train new models remains extremely high. There is broad recognition both in computer and domain sciences that generating training data remains the major bottleneck when developing ML models for targeted tasks. Increasingly, computer scientists are not attempting to produce highly-optimized models from general annotation frameworks, instead focusing on adaptation strategies to tackle new data challenges. Taking inspiration from large language models, computer vision researchers are now thinking in terms of “foundation models” that can yield reasonable zero- and few-shot detection and segmentation performance with human prompting. Here we consider the utility of this approach for ocean imagery, leveraging Meta’s Segment Anything Model to enrich ocean image annotations based on existing labels. This workflow yields promising results, especially for modernizing existing data repositories. Moreover, it suggests that future human annotation efforts could use foundation models to speed progress toward a sufficient training set to address domain specific problems. 

Abstract The ocean is experiencing unprecedented rapid change, and visually monitoring marine biota at the spatiotemporal scales needed for responsible stewardship is a formidable task. As baselines are sought by the research community, the volume and rate of this required data collection rapidly outpaces our abilities to process and analyze them. Recent advances in machine learning enables fast, sophisticated analysis of visual data, but have had limited success in the ocean due to lack of data standardization, insufficient formatting, and demand for large, labeled datasets. To address this need, we built FathomNet, an open-source image database that standardizes and aggregates expertly curated labeled data. FathomNet has been seeded with existing iconic and non-iconic imagery of marine animals, underwater equipment, debris, and other concepts, and allows for future contributions from distributed data sources. We demonstrate how FathomNet data can be used to train and deploy models on other institutional video to reduce annotation effort, and enable automated tracking of underwater concepts when integrated with robotic vehicles. As FathomNet continues to grow and incorporate more labeled data from the community, we can accelerate the processing of visual data to achieve a healthy and sustainable global ocean. 

Imaging is increasingly used to capture information on the marine environment thanks to the improvements in imaging equipment, devices for carrying cameras and data storage in recent years. In that context, biologists, geologists, computer specialists and end-users must gather to discuss the methods and procedures for optimising the quality and quantity of data collected from images. The 4^thMarine Imaging Workshop was organised from 3-6 October 2022 in Brest (France) in a hybrid mode. More than a hundred participants were welcomed in person and about 80 people attended the online sessions. The workshop was organised in a single plenary session of presentations followed by discussion sessions. These were based on dynamic polls and open questions that allowed recording of the imaging community’s current and future ideas. In addition, a whole day was dedicated to practical sessions on image analysis, data standardisation and communication tools. The format of this edition allowed the participation of a wider community, including lower-income countries, early career scientists, all working on laboratory, benthic and pelagic imaging. This article summarises the topics addressed during the workshop, particularly the outcomes of the discussion sessions for future reference and to make the workshop results available to the open public.